A block storage system uses protocols, such as small computer system interface (SCSI) and advanced technology attachment (ATA), to access blocks of data. The block storage system may use caching and/or tiering to more efficiently access the blocks of data. The block storage system also may use a virtual addressing scheme for provisioning, de-duplication, compression, caching, tiering, and/or providing resiliency of data stored on different physical media through methods such as replication and migration. Virtual addressing allows the user of the storage system to access blocks of data on the storage system while allowing the storage system administrator to manage media count and types, access methods, redundancy and data management features without the users' knowledge.
A file storage system manages the data blocks and metadata associated with different files. Files can have variable sizes and may include metadata identifying the associated data blocks. A user of a file storage system may access files or portions of files whereas the metadata is typically managed by and only accessed by the file storage system. The file storage system may de-duplicate, compress, cache, tier and/or create snapshots of the file data. An object storage system uses handles to put or get objects, usually in their entirety, from object storage. Object storage systems can perform timeouts, scrubbing, caching and/or checkpoints on the stored objects. The file storage system may operate on top of the block storage system and the object storage system either may operate on top of the block storage system or operate on top of the file storage system. A user of an object storage system may access objects whereas the underlying block or file storage is typically managed by and only accessed by the block or file storage system.
Clients may access data differently and thus have different storage requirements. For example, a first user may perform transactional operations that read and write data into random storage locations. A second user may perform analytic operations that primarily read large blocks of sequential data. In such a case, the performance of the first user may be limited by the number of random operations of the storage system while the performance of the second user may be limited by the bandwidth capability of the storage system.
For example, the first user may need to recover data after a hardware or software failure. The storage administrator may configure a redundancy storage extension for the storage system, such as redundant array of independent disks (RAID) that strips the same data on multiple different disks.
The second analytic user may not need data redundancy. However, the redundancy storage extension is used throughout the storage system regardless of which user accesses the disks. Overall storage capacity is unnecessarily reduced since redundant backup data is stored for all users.
In a further example, a first user requiring redundancy and a second user requiring highest write performance both need access to the same data with some of that data accessed concurrently by both users. If the required data exists upon the same virtual storage (the same Logical Unit Number—LUN), the storage administrator will be required to configure the entire LUN for the redundancy requirement of the first user which reduces the performance for the second user.
The storage administrator also may configure a caching or tiering policy that uses random access memory (RAM) and/or Flash memory to increase access rates for the random read and write operations performed by the first user. The caching or tiering policy is commonly applied to the entire storage system for all storage accesses by all users and minimally to all users of the particular storage data. As an example, if a block storage system enables caching for a particular disk (virtual or physical), said caching is enabled and functions equivalently for all clients accessing said storage data.
The caching or tiering policy may increase data access speeds for the first user but may provide little improvement for the large sequential read operations performed by the second user. Applying the caching or tiering policy to all storage operations may actually reduce storage performance. For example, data from large sequential read operations performed by the second user may flush data from RAM or Flash memory currently being cached or tiered for the first user. Additionally, read accesses from a non-benefitting user will evict data within the cache to the likely detriment of the benefitting user, thereby reducing performance for both users.